Generalization bounds of ERM algorithm with V-geometrically Ergodic Markov chains

The previous results describing the generalization ability of Empirical Risk Minimization (ERM) algorithm are usually based on the assumption of independent and identically distributed (i.i.d.) samples. In this paper we go far beyond this classical framework by establishing the first exponential bound on the rate of uniform convergence of the ERM algorithm with Vgeometrically ergodic Markov chain samples, as the application of the bound on the rate of uniform convergence, we also obtain the generalization bounds of the ERM algorithm with V-geometrically ergodic Markov chain samples and prove that the ERM algorithm with V-geometrically ergodic Markov chain samples is consistent. The main results obtained in this paper extend the previously known results of i.i.d. observations to the case of V-geometrically ergodic Markov chain samples. Communicated by Ding Xuan Zhou. This work is supported in part by National 973 project (2007CB311002), NSFC key project (70501030), NSFC project (61070225), FHEC (Q20091003) and China Postdoctoral Science Foundation (20080440190, 200902592). B. Zou (B) Faculty of Mathematics and Computer Science, Hubei University, Wuhan, 430062, China e-mail: [email protected] B. Zou · Z. Xu · X. Chang Institute for Information and System Science, Faculty of Science, Xi’an Jiaotong University, Xi’an, 710049, China Z. Xu e-mail: [email protected] X. Chang e-mail: [email protected]